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Abstract — In this work, we give good concatenated code 
ensembles for the binary erasure channel (BEC). In particular, 
we consider repeat multiple-accumulate (RMA) code ensembles 
formed by the serial concatenation of a repetition code with 
multiple accumulators, and the hybrid concatenated code (HCC) 
ensembles recently introduced by Roller et al. (5th Int. Symp. 
on Turbo Codes & Rel. Topics, Lausanne, Switzerland) consisting 
of an outer multiple parallel concatenated code serially concate- 
nated with an inner accumulator. We introduce stopping sets 
for iterative constituent code oriented decoding using maximum 
a posteriori erasure correction in the constituent codes. We 
then analyze the asymptotic stopping set distribution for RMA 
and HCC ensembles and show that their stopping distance 
/imin, defined as the size of the smallest nonempty stopping set, 
asymptotically grows linearly with the block length. Thus, these 
code ensembles are good for the BEC. It is shown that for RMA 
code ensembles, contrary to the asymptotic minimum distance 
dmin, whose growth rate coefficient increases with the number of 
accumulate codes, the hmin growth rate coefficient diminishes 
with the number of accumulators. We also consider random 
puncturing of RMA code ensembles and show that for sufficiently 
high code rates, the asymptotic /i m i n does not grow linearly 
with the block length, contrary to the asymptotic d m m, whose 
growth rate coefficient approaches the Gilbert- Varshamov bound 
as the rate increases. Finally, we give iterative decoding thresholds 
for the different code ensembles to compare the convergence 
properties. 

Index Terms — Asymptotic stopping set distribution, binary 
erasure channel, EXIT charts, hybrid concatenated codes, repeat 
accumulate codes, spectral shape function, stopping set, uniform 
interleaver. 

I. Introduction 

Turbo codes introduced by Berrou et al. in [1], and Gal- 
lager's low-density parity-check (LDPC) codes [2], rediscov- 
ered by MacKay in [3], are considered among the most 
powerful error-correction schemes of today due to their low 
decoding complexity and near-capacity performance on a wide 
variety of channels. Recently, serially concatenated codes, 
such as very simple repeat accumulate (RA) codes, and hybrid 
concatenated codes (HCCs), i.e., mixed parallel and serial 
structures combining the features of the two concatenations 
[4-13], have also attracted some attention, since they yield 
better minimum distances than turbo codes. 

For repeat multiple-accumulate (RMA) codes, Pfister and 
Siegel [5] showed that the minimum distance d m ; n increases 
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as the number of accumulators increase. In the limit of 
infinitely many accumulators, it was shown in [5] that the d m ; n 
approaches the Gilbert- Varshamov bound (GVB). Recently, 
Kliewer et al. [10] and Fagnani and Ravazzi [11, 12] showed 
independently that RMA code ensembles are asymptotically 
good, in the sense that their typical d m i n asymptotically grows 
linearly with the block length. A method for the calculation 
of a lower bound on the growth rate coefficient was also 
given in [10]. The same principle was applied in [8,9] to 
HCC structures. In this work, when we speak about HCCs we 
mean the HCC structures from [8], even if this is not explicitly 
stated. 

In iterative decoding for the binary erasure channel (BEC), 
stopping set distributions play an analogous role to that of 
the distance spectra in maximum-likelihood (ML) decoding. 
Stopping sets for iterative belief-propagation (BP) decoding 
of LDPC codes were introduced by Di et al. in [14]. In [15], 
Rosnes and Ytrehus adapted the concept of stopping sets to 
turbo decoding and introduced turbo stopping sets. Also, an 
exact condition for decoding failure on the BEC was stated as 
follows. Apply turbo decoding until the transmitted codeword 
has been recovered, or the decoder fails to progress further. 
Then the set of erased positions that remain when the decoder 
stops is equal to the unique maximum-size turbo stopping set 
which is also a subset of the (initial) set of erased positions. 
The stopping set concept has also been adapted to iterative 
row-column decoding of product codes in a recent paper [16]. 

In this work, we adapt the concept of stopping sets to 
iterative constituent code oriented decoding using maximum a 
posteriori (MAP) erasure correction in the constituent codes 
of RMA codes and HCCs. We then give expressions for their 
average stopping set distributions and analyze their asymptotic 
behavior. We show that both RMA and HCC ensembles, 
consisting of an outer multiple parallel concatenated code 
(MPCC) serially concatenated with an inner accumulator [8], 
exhibit an asymptotic stopping distance h m i n , defined as the 
size of the smallest nonempty stopping set [17], that grows 
linearly with the block length. Therefore, these code ensembles 
are good for the BEC. For RMA code ensembles, contrary to 
the asymptotic d m i n , whose growth rate coefficient increases 
with the number of accumulate codes, the asymptotic h m [ n 
growth rate coefficient diminishes with the number of accu- 
mulators. We also consider random puncturing of the RMA 
code ensemble and show that for sufficiently high code rates, 
the asymptotic ft, m j n does not grow linearly with the block 
length, contrary to the asymptotic d m i n , whose growth rate 
coefficient approaches the GVB as the rate increases [10]. 

The reminder of this paper is organized as follows. The 
encoder structures of RMA codes and the HCCs from [8] are 
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Fig. 1. Encoder structure for RMA codes. 



described in Section II. In Section III, we describe iterative 
constituent code oriented decoding using MAP erasure cor- 
rection in the constituent codes of concatenated codes and 
introduce stopping sets for this particular iterative decoding 
algorithm. We further give some of the basic properties and 
show that these stopping sets characterize exactly the perfor- 
mance of iterative constituent code oriented decoding on the 
BEC. Section IV discusses stopping set enumerators for RMA 
and HCC ensembles. Also, a factor graph interpretation is 
given. Section V introduces the asymptotic stopping set size 
spectral shape function. In Sections VI and VII a stopping 
distance analysis is performed for RMA code ensembles 
and HCC ensembles, respectively. Convergence properties are 
studied in Section VIII, where an extrinsic information transfer 
(EXIT) charts analysis is performed. Conclusions are given in 
Section IX. 

II. Encoder Structures 

The encoder structure of RMA codes is depicted in Fig. 1 . 
It is the serial concatenation of a repetition code Co of rate 
Rq = 1/q with the cascade of L > 1 identical rate- 1, memory- 
one, accumulators Cj, I = 1, . . . , L, with generator polynomi- 
als g(D) = 1/(1 + D), through interleavers 7Ti, . . . ,ttl. The 
overall nominal code rate (avoiding termination) is denoted by 
R = K/N = 1/q, where K and N denote the input and the 
output block length, respectively. Higher rates may be obtained 
by puncturing the output of the most inner accumulator Cl- 

We also consider the HCC ensembles introduced in [8] 
consisting of an outer MPCC with q parallel branches C\, 

1 = 1, . . . , q, serially concatenated with an inner accumulator, 
denoted by C q +x. Four different encoder structures, depicted 
in Fig. 2, are considered. For type-1 and type-2 codes, all the 
code bits from the outer MPCC enter the inner accumulator, 
while for type-3 and type-4 codes only q — 1 of the q 
parallel branches enter the inner accumulator. The nominal 
code rate is denoted by R = K/N. For type-1, type-3, and 
type-4 HCCs the code rate is R = 1/q, while for type- 

2 HCCs R = 1/(51+92) = 1/q, where qi denotes the 
number of feedforward branches and (72 denotes the number 
of recursive branches. The outer MPCC of type-2 and type- 

3 HCCs generalizes the R = 1/4 MPCC introduced in [18], 
which incorporates a feedforward branch (g{D) — 1+D) since 
it yields better convergence behavior than a MPCC with only 
recursive branches. Note also that in the type-4 HCC encoder 
Ci performs an identity mapping, i.e., the HCC is systematic. 
In [8], the asymptotic and finite-length d m - m properties of these 
four encoders were studied in detail for q = 4. Furthermore, 
iterative decoding thresholds on the additive white Gaussian 
noise (AWGN) channel were estimated using EXIT charts. In 
terms of d m i n properties, type-1 codes are the best and type-4 
codes the worst, while in terms of iterative decoding thresholds 
on the AWGN channel the ranking is opposite [8]. 



III. Iterative Constituent Code Oriented 
Decoding and Stopping Sets 

In this paper, we consider stopping sets for iterative de- 
coding of concatenated codes. Thus, we assume that the 
concatenated codes in Figs. 1 and 2 are decoded iteratively 
in a constituent code oriented fashion using MAP constituent 
decoders over the BEC. By iterative constituent code oriented 
decoding we mean a decoding strategy that iterates between 
the constituent decoders, i.e., the turbo-decoding principle 
applied to more general concatenated codes. MAP decoding 
of the constituent codes can be implemented efficiently on a 
trellis representation of the constituent code [15, 19]. The aim 
of trellis-based iterative constituent code oriented decoding on 
the BEC is to find a set of paths through each constituent 
code trellis that is consistent with the received sequence. 
The decoding starts with a set of all paths and iteratively 
eliminates those that are inconsistent. This iterative process 
continues until either there is only one possible path left in 
each constituent trellis (successful decoding), or there is no 
change from one iteration step to the next [15]. The complexity 
of iterative constituent code oriented decoding of concatenated 
codes is linear in the block length N when the number of 
constituent code activations is independent of N. 

As an alternative to trellis-based decoding, decoding of 
the constituent codes could be based, e.g., on a factor graph 
representation, but this will not necessarily be MAP erasure 
correction. Note that if the factor graph is constructed based 
on all codewords of the dual code, or if the factor graph does 
not contain any cycles, then the peeling decoder of Luby et 
al. [20] implements MAP erasure correction in the constituent 
codes [2 1 , 22] . Indeed, for the stopping set analysis carried out 
in this paper, we do not require to make any assumption on 
the decoding algorithm used at each constituent decoder. The 
key point is that decoding is performed in a constituent code 
oriented fashion, and that the constituent decoders perform 
MAP decoding. However, we prefer to refer to trellis-based 
decoding, since the trellis representation of the constituent 
encoders is very useful for the derivation of the stopping set 
enumerators and the subsequent asymptotic analysis. This will 
become apparent in the following sections. 

A. Stopping Sets for RMA Codes 

We will now give the formal definition of a stopping set 
for RMA codes, adapted from the definition in [15] for turbo 
stopping sets. The generalization to the HCC ensembles of 
Fig. 2 and to the case with puncturing is straightforward. In 
the following stopping set definition, we need the concept of 
support set of a binary vector and of a binary linear code. 
The support set x( x ) of a binary vector x = (x\, . . . ,xn) (of 
length N) is the set of nonzero coordinates. As an example, 
with x = (0,1,1,0,1), x(x) = {2,3,5}. The support set 
x(C) of a binary linear code C is the union of the support 
sets of each codeword in C. Also, we need the concept of a 
subcode. A subcode C of a binary linear code C is a subspace 
of C. Finally, an interleaver will be regarded as a mapping 
from the set of coordinates of its input sequence to the set of 
coordinates of its output sequence. 
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Fig. 2. Encoder structures for HCCs. 
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Definition 1: Let Crma denote a given RMA code with in- 
terleaves 7Tl, . . . , 7TJ,. A Set 5 = iS(7Tl, . . . , 7Tl) C {1, . . . , iV} 

of the coordinates of the output sequence (or codeword) is a 
stopping set if and only if there exist L + 1 linear subcodes 
G C Ci C {0, 1}^, l = 0,...,L, with support sets x(Q) 
such that 

X(C L ) = S and Mx(Ci-i)) = X (Ci), I = 1, ■ ■ ■ ,L. 

The size of a stopping set S is its cardinality. 

Note that in Definition 1 we used the fact that the mapping 
between the input support set and the output support set is 
an identity mapping for rate R = 1 encoders. For general 
R < 1 encoders this is not true. Also, note that Definition 1 
does not exclude the empty set. Thus, the empty set is 
formally a stopping set of size zero. The size of the smallest 
nonempty stopping set is called the stopping distance [17] and 
is denoted by /i m i„. We emphasize the fact that the concept of 
stopping sets for RMA codes, as defined above, is conceptually 
different from the traditional concept of stopping sets used 
in connection with iterative BP decoding of LDPC codes, 
but it reduces to the traditional concept of stopping sets for 
Tanner graphs when the constituent codes are single parity- 
check codes. Indeed, the RMA code could be decoded using 
iterative BP decoding on a Tanner graph representing the entire 
code [23]. On the other hand, the concept of stopping sets from 
Definition 1 is related to iterative constituent code oriented 
decoding. In general, the concept of stopping sets can be 
defined for any iterative decoding algorithm when it operates 
on the BEC and should not be used without having a specific 
iterative decoding algorithm in mind. 

In general, the erasure probability after iterative BP decod- 
ing of a concatenated code using its factor graph is greater than 
or equal to the erasure probability after iterative constituent 
code oriented decoding using MAP erasure correction in the 
constituent codes for any channel erasure pattern. Here, with 
the factor graph of a general concatenated code we mean a 
factor graph constructed from arbitrary Tanner graphs of each 



constituent code. The Tanner graphs of the constituent codes 
are interconnected through the interleavers. An example of 
such a factor graph is given in Fig. 4 for a repeat accumulate- 
accumulate (RAA) code with q = 3. The interested reader is 
referred to [15] for further details. For the special case of RMA 
codes and HCCs with accumulate constituent encoders, it turns 
out that iterative BP decoding on the overall factor graph is 
equivalent to iterative constituent code oriented decoding. This 
is because the factor graph of an accumulator (depicted in 
Fig. 4) does not contain cycles [24, p. 583]. Thus, iterative 
BP decoding of a constituent accumulator (using a factor 
graph) is equivalent to MAP erasure correction. Actually, this 
result holds for any accumulator with generator polynomial 
1/(1 + £)*), t > 1. However, we stress that the traditional 
definition of stopping sets (on an overall factor graph) is only 
appropriate when the constituent encoders are accumulators, 
but not for general concatenated codes, i.e., when other con- 
volutional codes are used as constituent codes. This is one of 
the reasons why we have introduced iterative constituent code 
oriented decoding and the corresponding general definition of 
stopping sets in Definition 1. Indeed, an asymptotic stopping 
distance analysis of the concatenated codes analyzed here 
using, e.g., 4-state or 8-state convolutional encoders could be 
carried out using the approach in [25] to compute asymptotic 
input-output weight distributions of convolutional encoders, 
generalized to the stopping set case. 

In summary, the stopping set enumerators and the analysis 
in the following sections are general and do not only apply 
to iterative constituent code oriented decoding; iterative BP 
decoding on the overall factor graph of RMA codes and the 
HCCs of Fig. 2 leads to the same stopping set enumerators as 
iterative constituent code oriented decoding. 

The following lemma can easily be proved. 

Lemma 1: Let C denote a RMA code (or a HCC). Then, 
the support set of any codeword in C is a stopping set. 

Theorem 1: Let C denote a RMA code (or a HCC) that 
we use to transmit information over the BEC. The received 
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vectors are decoded iteratively using constituent code oriented 
decoding using MAP erasure correction in the constituent 
codes until either the codeword has been recovered, or the 
decoder fails to progress further. Then the set of erased 
positions that remain when the decoder stops is equal to the 
unique maximum-size stopping set that is contained in the 
(initial) set of erased positions. 

Proof: The proof uses the same basic ideas as the proof 
of Theorem 1 in [15], and is omitted for brevity. ■ 
From Theorem 1 it arises that an important parameter 
for code performance is the stopping distance h m i D . In the 
following sections we address the asymptotic behavior of /i m ; n 
for RMA and HCC ensembles. 

IV. Support Sets and Stopping Set Enumerators 

Let C denote an (N, K) binary linear code. Partition all the 
subcodes of C of dimension d, d = 0, . . . , K, into equivalence 
classes based on their support sets. In particular, all subcodes 
within a specific subcode class are required to have the same 
support set, but the subcodes may have different dimensions. 
We define the subcode input-output support size enumerating 
function (SIOSEF) [15] of C as 



K N 



W) = E2> : 



where W and H are dummy variables, and h is the number 
of subcode classes of C of input support set size w and output 
support set size h. In the rest of the paper, with a slight abuse 
of notation, we will refer interchangeably to both A c (W, H) 
and a% h as the SIOSEF of a code C. 



A. Stopping Set Enumerators for RMA and HCC Ensembles 

Let h be the ensemble-average input-output stopping set 
size enumerating function (IOSSEF) of the code ensemble C 
with input and output block length K and N, respectively, 
denoting the average number of stopping sets of input size w 
and output size h over C. Also, denote by = >~2w=o *\v h 
the ensemble-average stopping set size enumerating function 
(SSEF) of the code ensemble C, giving the average number of 
stopping sets of size h over C. 

Using the concept of uniform interleaver [26], the IOSSEF 
of a RMA code ensemble Crma can be obtained from the 
SISOEFs of the constituent encoders. Indeed, the contribution 
of the SISOEFs of the constituent encoders to the IOSSEF of 
a RMA code ensemble is, through Definition 1, analogous to 
the role played by the weight spectra of constituent encoders 
in the weight spectrum of a concatenated code ensemble. The 
ensemble-average IOSSEF of Crma can then be written as 
[15,26] 
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Fig. 3. Extended trellis module of the rate-1 recursive convolutional encoder 
with generator polynomial g(D) = 1/(1 + D). The edge labels have the 
format input/output. 



where sj 1 "* h h is called the conditional support size 
enumerating function of Crma- 

Now, consider the (TV, K) HCC ensembles Chcc of Fig. 2. 
Assuming trellis termination, the Zth constituent code is a 
binary linear code with input block length Ki and output block 
length Ni. Except for C\, which is directly connected to the 
input, every code is preceded by an interleaver. Furthermore, 
we assume that the code C q+ \ is directly connected to the 
channel. Partition the set {1, 2, . . . , q} into two disjoint subsets 
Q and its complement Q such that Q contains the indices of all 
codes directly connected to the channel. The ensemble-average 
IOSSEF of Chcc can be written as [8, 15] 
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where s^f^ h h is called the conditional support size 
enumerating function of Chcc- 

The evaluation of (1) and (2) requires the computation of 
the SIOSEFs of the constituent encoders. In the following, 
we give closed-form expressions for the SIOSEFs of rate-1, 
memory-one, encoders, and repeat codes. They will be used 
later to derive the asymptotic expressions for the stopping set 
distributions in Sections VI and VII. 



B. SIOSEFs for Memory-One Encoders and the Repetition 
Code 

Theorem 2: The SIOSEF for rate-1, memory-one, convolu- 
tional encoders with generator polynomials g{D) = 1/(1 + D) 
and g(D) = 1 + D that are terminated to the zero state at the 
end of the trellis and with input and output block length 
can be given in closed form as 



LtJ 

d=l 



a w,h — a h,w 
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h-d 



d-1 \w-2d 



(3) 



for positive input sizes w. Also, Oq + d = Oq^ = 1. 

Proof: Let C N denote the binary linear code obtained by 
terminating (to the zero state) the rate-1 recursive convolu- 
tional encoder with generator polynomial g(D) = 1/(1 + D) 
and with input and output block length of N. In Fig. 3, 



the extended trellis module [15], denoted by % xt , of the 
encoder is depicted. We call the trellis module in Fig. 3 the 
extended trellis module because it extends (and comprises) 
the trellis module of the code to represent all support vectors 
of subcodes. Each edge between two trellis states is labeled 
with a binary pair a/b, where a denotes the input support 
vector bit and b denotes the output support vector bit. The 
overall transition between two states s, and s is denoted by 
(sj, a/b, s ). Note that T ext is identical to the trellis module of 
the original code, except that it has an extra edge (with label 
1/1) from state one to state one. The paths of length N starting 
from the zero state and ending in the zero state in an TV- 
fold concatenation of T cx t give all possible support vectors of 
subcodes of C N [15]. Also, the TV-fold concatenation of T ext 
does not contain duplicates, i.e., there are no two paths with 
the same input/output label sequence. From [27], we know 
that the number of paths in an iV-fold concatenation of T ext 
corresponding to codewords, i.e., paths that do not contain the 
(1, 1/1, 1) transition, of even input weight w > 2 (there are 
no codewords of odd input weight) and output weight h is 

N -h\/ h - 1 
w/2 ) \w/2 - 1 

These codewords consist of w/2 error events with a total 
of h — w/2 transitions from state one to state one. Now, 
the number of paths in the extended trellis, i.e., the TV-fold 
concatenation of T cxt , consisting of d error events with a total 
output weight of h and total input weight of w > 2 is 



N -h\fh-l 
d )(d-l 



h-d 
w-2d 



since the transition (1,1/1,1) does not increase the output 
weight (b is 1 for both edges between state one and state 
one), but only the input weight. The result for the encoder 
g(D) = 1/(1 + D) follows by summing over all possible 

values of d. Furthermore, a^ D = 1, since the empty set is 
the support set of the trivial subcode containing only the all- 
zero codeword. The SIOSEF for the feedforward encoder with 
generator polynomial g(D) = 1 + D is obtained in a similar 
manner. ■ 

Note that the formula in (3) generalizes the closed-form 
expression for the input-output weight enumerating function 
for rate-1, memory-one, convolutional encoders from [27]. 

Theorem 3: The SIOSEF for the (Kq, K) repetition code 
Co with input block length K obtained by concatenating 
together K successive codewords of a (q, 1) repetition code 
can be given in closed form as 



,Co 



(4) 



Proof: Consider the (q, 1) repetition code C. This code 
consists of two codewords, the all-zero codeword and the all- 
one codeword. Obviously, A C (W, H) = 1 + WH q . With 
an input block length of K, we get (1 + WH q ) K = 
E^Lo Q W w m w , from which it follows that a%° qw = 
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Fig. 4. Factor graph representation of a RAA code with q = 3. 
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Fig. 5. Factor graph representation of an extended trellis section of an 
accumulate code. 



C. A Factor Graph Interpretation 

In this section, we consider an equivalent factor graph 
representation of the accumulate code to better clarify the 
computation of the SIOSEF and its connection with stopping 
sets of a concatenated code. The factor graph of a RAA code 
with q = 3 is depicted in Fig. 4. Consider a single section of 
the factor graph corresponding to one of the two accumulate 
codes, marked with a dashed box, and transmission over the 
BEC. We have the relation uu = Xk-i + Xk, where uu is the 
information symbol, and Xk-i and Xk are the code symbols 
of section k. The decoder will fail in recovering an erased 
symbol at section k if and only if two or more symbols 
at section k are erased. Four possible triplets (ufc, Xk-i, Xk) 
are possible, namely (1,0,1), (1,1,0), (0,1,1), and (1,1,1), 
where 1 means that the symbol is erased. Their factor graph 
representation is given in Fig. 5, where a black circle means 
that the symbol is erased (1) and an empty circle means 
that the symbol is not erased (0). Clearly, to enumerate the 
stopping sets of an accumulate code of length N we must 
consider all valid combinations (of length N) of the sections 
in Fig. 5 plus the one corresponding to the triplet (0, 0, 0) (no 
erasures). Note that we do not need to consider the triplets 
with a single one, since in this case the erasure would be 
recovered, yielding to (0,0,0). To simplify this computation, 
the five factor graph sections of Fig. 5 can be equivalently 
represented in a more compact form by T cxt in Fig. 3, which 
gives a trellis representation of all possible support vectors of 
subcodes of an accumulate code C. The five triplets (0,0,0), 
(1,0,1), (1,1,0), (0,1,1), and (1,1,1) of Fig. 5 correspond 
to transitions (0,0/0,0), (0,1/1,1), (1,1/0,0), (1,0/1,1), 
and (1,1/1,1), respectively, in T cxt . The enumeration of all 
stopping sets, or all support sets of subcodes, of the accumulate 

code, given by a^~? , can then be computed using % xt as 
shown in the previous subsection. Please note that here, for 
better understanding, we used the concept of stopping sets 
applied to an accumulate code. Formally speaking, stopping 
sets, as defined in Section III-A, apply only to iterative decod- 
ing of concatenated codes. When referring to the constituent 
codes (with the associated MAP decoding algorithm) it is more 
appropriate to use the concept of support sets discussed in 
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Section III- A. 

Finally, the IOSSEF of RMA and HCC ensembles can be 
obtained by properly combining the SIOSEFs of the con- 
stituent encoders through (1) and (2), respectively. 

V. Finite-Length and Asymptotic Analysis of 
Stopping Sets 

A. Finite-Length Analysis of Stopping Sets 

The ensemble-average SSEF of a concatenated code 
ensemble C can be used to bound its /i m ; n in the finite-length 
regime, similarly to the case of the c? m ; n [2,5]. In particular, 
the following bound holds. 

Lemma 2: The probability that a code chosen randomly 
from the ensemble C with ensemble-average SSEF s£ has 
stopping distance /i m ; n < h is upper-bounded as 

Pr(/i min < K) < 4- (5) 
h=l 

Proof: The bound follows from the application of the 
union bound and the Markov inequality [5]. ■ 
Lemma 2 can be used to obtain a probabilistic lower bound on 
the stopping distance of a code ensemble. In particular, if we 
set Pr(/i m ; n < h) = s, where e is any positive value between 
and 1, we would expect that at least a fraction 1 — e of 
the codes in the ensemble have a stopping distance /i m ; n of at 
least h. 

B. Asymptotic Analysis of Stopping Sets 

We define the asymptotic stopping set size spectral shape 
function as [2] 

r s (p) = ^lim oo Sup^ln^ JVJ (6) 

where sup(-) denotes the supremum of its argument, p = 
is the normalized stopping set size, and N is the block length. 
From (6), the SSEF can be expressed as ~ c NTs ^ when 
N — ► oo. Therefore, if there exists some abscissa p > 
such that sup p<p , r s (p) < Vp* < po, and r s (p) > 
for some p > po, then it can be shown (using Lemma 2 
for example) that, with high probability, the stopping distance 
of most codes in the ensemble grows linearly with the block 
length N, with growth rate coefficient of at least po. On the 
other hand, if r s (p) is strictly zero in the range (0,po)> it 
cannot be proved directly whether h m i n grows linearly with 
the block length or not. 

In [11, 12], it was shown that the spectral shape function of 
RMA codes, for the codeword case, exhibits this behavior, 
i.e., it is zero in the range (0,po) an d positive for some 
p > po, where p means here the normalized output weight. 
By combining the asymptotic spectral shapes with the use of 
bounding techniques, Fagnani and Ravazzi were able to prove 
in [11, Theorem 1 1 ] that the minimum distance of RMA codes 
indeed grows linearly with the block length with growth rate 
coefficient of at least po- In Sections VI and VII we show that 
the stopping set size spectral shape function of RMA and HCC 
ensembles exhibits a similar behavior. We shall then extend 



Theorem 11 in [11] to prove that RMA codes and HCCs are 
also good for the BEC. 

We remark that in the rest of the paper, with a slight 
abuse of language, we sometimes refer to po as the exact 
value of the asymptotic growth rate coefficient. However, we 
emphasize that, strictly speaking, p is only a lower bound on 
the asymptotic growth rate coefficient. 

VI. Stopping Distance Analysis for RMA Codes 

In this section, we analyze the asymptotic stopping distance 
properties of RMA code ensembles. We first show that the 
hmin of RA code ensembles cannot grow linearly with the 
block length. Then, we consider the ensemble of RMA codes 
obtained from the concatenation of a repeat code with the 
cascade of L > 1 accumulators and show that the asymptotic 
h m in grows linearly with the block length. 

A. RA Codes 

We consider the /i m ; n of code ensembles formed by the 
concatenation of a R = 1/q repetition code with a single 
inner accumulator (L = 1) with generator polynomial g{D) = 
1/(1 + D) through a uniform interleaver [26] (see Fig. 1). In 
particular, we prove the following theorem. 

Theorem 4: The stopping distance /i^-* of a RA code 
ensemble Cra with repetition factor q > 3 satisfies 

lim Prte<^ £ )=0 

where e is any positive constant. 

Proof: Using (1), (3), and (4), the ensemble-average 
IOSSEF s^f with w > of a RA code ensemble can be 
written as 

fifNv^L^rJ (N-h\(h—l\( h-d \ 
_C RA _ \w) 2-<d=l \ d )\d-l)\qw-2d) 
S wJi " /N\ 
\qwj 

Using Stirling's approximation (?) < ( 2 |-)' C and the fact that 

nU* - o > with <px(t) - cx P l^i) [io], t RA h 

can be bounded as 

w,h — 

I S£ I 

(qw)\e c i w + w ' 1 (p N (qw- 1) (qw - 2d) 2d -i w 

q w w w 2_, d d (d-l) d - 1 

^ v j 

g(w,N) 

(7) 

The total average number of nonempty stopping sets of size 
h < h can now be obtained from s^ R ,* <n = 2~2h=i ^w t- Using 
(7), we get 

s C w R h<n<N w -\^h^ + l^\g(w,N). (8) 

Let = Eli t K h<H = £™=? fe» where we used 

the fact that w < 2h/q. This follows from the binomial 
coefficient ( q ^Z-2d) ' f rom which it follows that qw < h + d < 
h + [qw/2\, which implies that w < 2h/q < 2h/q. Also, we 



7 



assume that h can be expressed in terms of the block length 
TV as H = N v , with < v < 1. From (8), it follows that 

2 



5 Cra 

s h<h 



< 



max 



N w ~ J +1 g(w, N) 

q l<w<2H/q 

q 

for large enough N. We choose such that 

lim N 1 -\^h q+ l%\ +1 g(l,N)=Q 

N >oo 



which implies that 
1 



~q~ 




q 


+ 




+ v(q + 




2 


.2. 





< 



(9) 



since g(l,N) approaches a constant when N approaches 
infinity. The evaluation of (9) leads to v < 3 g ~ 2 , and we can 

q-2 

express the stopping set size h as A 3 «+ 2 e for any positive 
constant e. Since (see Lemma 2 in Section V-A) 



Pr (h°£<h) 



< a 



^Cra 



h<h 



the theorem follows. ■ 
We remark that we can improve the lower bound of (q — 
2)/(3<7 + 2) from Theorem 4 using a much more convoluted 
approach. In particular, the lower bound can be tightened to 
(q — 2)/ (3g) for all q > 3 using a similar technique as in 
the proof of Theorem 1 in [10]. However, in the stopping set 
case, the summation over d in (7) needs to be upper-bounded 
by \_ 3 j-\ times the maximum element found by taking the 
derivative with respect to d. Further details are omitted for 
brevity. 

Corollary 1: In the ensemble of RA codes with repetition 
factor q > 3 almost all codes have stopping distance lower- 
bounded for N — ► oo by 

h c * A > N&~ e 

mm 

where e is any positive constant. 

An upper bound for the stopping distance is given by the 
following theorem. 

Theorem 5: The stopping distance ft^j-n of any code in the 
RA code ensemble with repetition factor q > 3 is upper- 
bounded for N — ► oo by 

h CR A <0 ( N ^ 

mm — \ 

Proof: The proof given in [28] for d m - m of RA code 
ensembles still holds for the stopping distance. ■ 
As a consequence of Theorems 4 and 5, the stopping 
distance of almost all codes in the RA code ensemble with 
q > 3 grows with TV as 0{N V ), where (q - 2)/(3q + 2) < 
v < (q — l)/q, i.e., the RA code ensemble is bad for the BEC. 

B. RMA Codes 

In the following, we consider the ensemble of codes formed 
by a rate R = l/q repetition code followed by the cascade of 
L accumulators, and show that the ensembles with parameters 
L > 3 and q > 2, and L = 2 and q > 3 are good for 
the BEC, in the sense that, asymptotically, their typical /i m ; n 



grows linearly with the block length. In particular, we need to 
prove that s h <"^ A tends to zero as N — ► oo for all h< PqN, 
for some value p between and 1. To prove this, we follow 
similar arguments to the ones used in [10] and [11, 12] for the 
asymptotic d m i n . 

Using (3) and (4) in (1), the conditional support size 
enumerating function (with w > 0) of RMA code ensembles 
can be written as 



w,hi,...,hL — i,h 



(K\ L^J (N-hi\ (hi-l\ ( hi-di \ 
\ w ) Z^d 1 = l \ d x )\di-l)\qw-2di) 

\qw/ 

i^i I 

L-lv^L 2 J (N-hi\(hi-\\( hi-di \ 
x -Q ^d l = l \ ,/; :'w... , >_,!,) 

1=1 



(hi-i) 



L 2 J IN-h\lh-\\l h-d L \ 
2~id L = l \ d L )\d L -l)\h L - 1 -2d L ) 



( N ) 



EE-- E 

d\ = l d2 = l dL = l 



w,hi,...,hL-i,di,...,dL,h' 



Without loss of generality, we can write 

w = aN a , hi = piN bi , i = l,...,L-l, 
h = pN c , and d t = ^ t N e% , i = 1, . . . , L 



(10) 



(11) 



where 0<a<6i<6 2 <---< b L -i <c<l,0<ei<a< 
1, and < a < bi-i < 1, i = 2, . . . , L. These inequalities 
can be derived from the binomial coefficients in the expression 
in (10) combined with the fact that for a binomial coefficient 
(fe). n > k > 0. Also, a,/3i, . . . and p are 

positive constants. We must consider two cases: 1) at least 
one of the quantities w, hi, . . . , hL-\,d\, . . . , d^, or h is of 
order o(N), and 2) all quantities w, h\, . . . , hL-i,d\, . . . , d^, 
and h can be expressed as fractions of the block length N, 
i.e., a — bi = ■■■ = = d\ = ■ ■ ■ = dL = c = 1. 

The following lemma addresses the first case for RAA code 
ensembles. 

Lemma 3: In the ensemble of RAA codes with block length 
N and q > 3, in the case where at least one of the quantities 
w, h lt di, d 2 , or h is of order o(N), N 6 ^^ di>da>h — > 
as N — ► oo for all values of h, 1 < h < N/2.' 

Proof: See Appendix A. ■ 

Lemma 3 can be generalized to the case of RMA codes 
with L > 3 (with or without puncturing) and to the HCC 
ensembles in Section VII. The proofs are omitted for brevity. 
As a consequence of Lemma 3, the contribution of the first 
case to s h ™ A tends to zero as V — ► oo, and we can assume 
that w, hi, ... , fiL-i,di, . . . , d^, and h are all linear in the 
block length. 

We now address the second case by deriving an expression 
for the stopping set size spectral shape function in (6) for 
RMA code ensembles. Using Stirling's approximation for the 
binomial coefficient (£) n -=^ e rM{k/n) where m ^ is the 
binary entropy function with natural logarithms, and the fact 
that w, hi, ... , hL-i,d\, . . . , dL, and h are all assumed to be 
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TABLE I 



/\i i Mr 1 U 




UKU W 1 ri KA 


n x? /^i rr r?rrT r'lc XT r 


rs for RMA Code Ensembles. 






q = 2 


9 = 3 


q=4 


q - 5 


q = 6 


PO (hmin) (L 


= 2) 


N/A 


0.0929 


0.1289 


0.1505 


0.1647 


PO (dmin) (L 


= 2) [10] 


N/A 


0.1323 


0.1911 


0.2286 


0.2549 


PO (fcmin) (i 


= 3) 


0.0681 


0.1037 


0.1194 


0.1279 


0.1331 


PO (dmin) (i 


= 3) [10] 


0.1034 


0.1731 


0.2143 


0.2428 


0.2643 


PO (hmin) (i 


= 4) 


0.0549 


0.0716 


0.0784 


0.0817 


0.0835 


GVB 




0.1100 


0.1740 


0.2145 


0.2430 


0.2644 



of the same order as the block length N 9 (10) can be written 

as 

n^RMA 

w,hi,...,hL — i,h 

= £- £ 

d 1= l d L =l 



exp{/(a,/3i, . . . ,/3l_i,7i, 
+ o(iV)} 



(12) 



oo. In (12), a = ^ is the normalized input 
■jSj* is the normalized output stopping 



when N 

stopping set size, p 

set size, /?; = 4^ is the normalized output support set size of 
constituent code C;, 7; = jj, and the function /(•) is given 
by 



H(A)) 



/=i 



■,1L,P) 



{01 



%-1 



+£# H I +£(a-tO h 



a)h 



7; 



-A 
27; 



A - 7i 



(13) 



where for conciseness we defined 0o — a and /3l — p- Finally, 
the stopping set size spectral shape function for RMA code 
ensembles can be written as [6] 

rf RMA (p)= sup f{0o, 01, • ■ • , 0L—i, 7i> ■ • • 1 7£i P) 

O<0o,...,Pl-i<1 
0<7i,...,7t<l 

(14) 

In deriving (14) from (12), we used the fact that max*(x, y) = 
In (exp(x) + exp(y)) is approximately equal to max(cc,y) 
when x and y are large and distinct. Similar comments apply 
when more than two variables are involved. 

From (13) and (14) it can easily be verified that the 
asymptotic stopping set size spectral shape functions of RMA 
code ensembles satisfy the recursive relation 



\p) 



sup 

0<u<l 



'KMA(l-l) 



(u) + p) 



where rs RMA(I) , I = L — 1,L, is the asymptotic stopping set 
size spectral shape function with I accumulators, ?~ S RMA(0) is 
defined to be the asymptotic stopping set size spectral shape 
function of a repeat code, and 



max(0,-u 

+ (l-p)H 



sup 

p)=7 = m i n (P:l~ 



-p.u/2) 
+ {P- 



-U{u 
7 )H 



+ pM 

u - 27 
p-7 



(15) 



0.14 



0.12 



0.10 



0.08 



,lL,p)N ^ 



0.04 



0.02 
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-0.02 
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u 
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3 jZ ^ 
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0.25 



Fig. 6. Asymptotic stopping set size spectral shape function for the RAA 
code ensemble with q = 2, 3, 4, 5, and 6. 



To analyze the asymptotic stopping distance behavior of 
RMA code ensembles, we must solve the optimization prob- 
lem in (13)-(14). Note that in (14) we did not include the 
constraints on the relationship of the variables involved in the 
function /(•), due to lack of space. However, these constraints 
must be considered in the optimization. The constraints on the 
involved variables can be derived by looking at the arguments 
of the binary entropy functions in the expression for the 
function /(•) in (13). In general, the argument of the binary 
entropy function should be between and 1. The maximization 
of the function /(•) is addressed in Appendix B. The numerical 
evaluation of (13)-(14) is shown in Fig. 6 for RAA code 
ensembles with q = 2,3,4,5, and 6. We observe that the 
stopping set size spectral shape function for the rate R = 1/2 
RAA code ensemble is strictly positive, meaning that the 
ensemble is bad for the BEC. For 3 < q < 6, the function 
rf™ A (p) is zero in the range (0,po) and positive for some 
values p > p . In this case, we cannot conclude directly 
whether /i m ; n grows linearly with the block length or not. 
However, we can prove the following theorem, extending the 
results in [11, 12] to the stopping distance case. 



Theorem 6: Define p$ = max{p* G [0, 1/2) 



Vp < p*}. Assuming that lim u _ 
then \fp* > 



-{P) = 



< Vp < p , 



lim Pr(h mia < ( Po -p*)N) = 

N >oo 

for L > 3 and q > 2, and L = 2 and q > 3. Thus, if p > 
and r^ nMA {p) > Vp (see Lemma 4 in Appendix B), then 
almost all codes in the ensemble have asymptotic stopping 
distance growing linearly with N with growth rate coefficient 
of at least pa. 

Proof: See Appendix C. ■ 
We remark that it can be verified that the assumption in 
Theorem 6 always holds for the numerical values of po that 
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TABLE II 

Asymptotic h min Growth Rate Coefficients for punctured RMA Code Ensembles with q = 3 and nominal code rate R' = l/(Xq). 



R' 




1/3 


0.35 


0.37 


0.38 


2/5 


5/12 


0.43 


4/9 


1/2 


0.54 


0.55 


X 




1 


20/21 


100/1 1 1 


50/57 


5/6 


4/5 


100/129 


3/4 


2/3 


50/81 


20/33 


pa (L = 


2) 


0.0929 


0.0911 


0.0885 


0.0868 


0.0820 


0.0746 


0.0673 


0.0585 


0.0240 


0.0028 


N/A 


pa (L = 


3) 


0.1037 


0.0866 


0.0632 


0.0514 


0.0289 


0.0124 


0.0015 


N/A 


N/A 


N/A 


N/A 


pa (L = 


4) 


0.0716 


0.0426 


0.0113 


N/A 


N/A 


N/A 


N/A 


N/A 


N/A 


N/A 


N/A 



we have found. From Lemma 3, Theorem 6, and the numerical 
evaluation of rf RAA (p) in Fig. 6, it results that for RMA codes 
the typical h m in asymptotically grows linearly with the block 
length with growth rate of at least po. The exact values of po 
are given in Table I. For comparison purposes we also give 
the asymptotic growth rate coefficient of the d m j n computed 
in [10]. As expected, the asymptotic growth rate coefficient of 
h m i n is smaller than for c? m i n . 

We can now prove the following theorem. 

Theorem 7: The typical /i m ; n of RMA code ensembles for 
L > 3 and q > 2 grows linearly with block length. 

Proof: Note that if we serially concatenate any encoder 
whose h m i n grows linearly with the block length with growth 
rate coefficient of at least po with an accumulate code through 
a uniform interleaver, the resulting concatenated code en- 
semble will exhibit a h m - m growing linearly with the block 
length with growth rate coefficient of at least |~po/2~|. This 
follows from the fact that the output support set size h of 
an accumulate code is lower bounded by |"^] in (3). In more 
detail, due to the binomial coefficient ( Z 2 d) > h — d > w — 2d, 
from which it follows that w < h + d < h + [w/2\, which 
implies that h > [^]. Since we know that the RAA code 
ensemble exhibits a typical h min that grows linearly with the 
block length, the theorem is proved for q > 3. For q = 2, 
it can be shown that the function rf RAAA (p) for the repeat 
triple-accumulate (RAAA) code ensemble is zero in the range 
(0, po), with po > 0, and positive for some p > po. Therefore, 
by repeating the argument for q > 3, the theorem is also 
proved for q = 2. ■ 

In Table I, we also report the asymptotic growth rate 
coefficient p for RAAA and repeat quadruple-accumulate 
(RAAAA) code ensembles. Interestingly, from Table I it 
follows that, contrary to the asymptotic d m in growth rate 
coefficient, which increases with the number of accumulators 
and tends to approach the GVB [10-12], the asymptotic 
growth rate coefficient of h m in decreases with the number of 
accumulators concatenated in series. An intuitive explanation 
to this behavior can be formulated as follows. From Fig. 3, 
it follows that there is a many-to-one mapping from input 
sequences to output sequences in an A^-fold concatenation of 
T ext , in the sense that for a given output sequence there are 
many input sequences that can produce it. This is not the case 
for the basic trellis module, i.e., for T ext in Fig. 3 without 
the transition (1,1/1,1), where the mapping is one-to-one. 
In a concatenation of L trellises based on the basic trellis 
module, there is still a one-to-one mapping between input 
sequences and output sequences. However, in a concatenation 
of L trellises based on T cxt , there will be an increasing set 
(in L) of input sequences that map to a particular output 
sequence. This phenomenon can be easily understood by 
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Fig. 7. Probabilistic lower bound on the stopping distance /i m i n versus block 
length N for RMA codes with q = 4 and L = 2, 3, and 4. 



considering the probabilistic bound (5) on hmin for the finite- 
length regime. Since the set of input sequences that map 
to a particular output sequence increases with L, the terms 



— 1 w.h 



in the right hand side of (5) will also 
increase. Consequently, h min will decrease. Our conjecture is 
that the same phenomenon applies to the asymptotic h m in 
growth rate coefficient. From these results, it is apparent that 
serially concatenated codes with more than three encoding 
stages are not well suited for the BEC. 

In Fig. 7, we plot the probabilistic lower bound on hmin 
from Lemma 2 for RMA codes with q = 4, L = 2, 3, and 
4, and codeword length N up to 1000 bits. The bounds were 
obtained by setting e = 0.5 in (5), i.e., at least half of the codes 
in the ensemble have stopping distance at least equal to the 
value indicated by the curves. The results are in agreement 
with the asymptotic h m in growth rates in Table I. The best 
growth rate is obtained for L = 2, while increasing the number 
of accumulate codes decreases the growth rate. 



C. RMA Codes with Random Puncturing 

In this section, we consider high rate RMA code ensem- 
bles obtained by puncturing the output of the most inner 
accumulator Cl- We assume that code bits at the output 
of Cl are punctured randomly, since otherwise we cannot 
guarantee asymptotic linear growth rate for h m in- Denote by A 
(0 < A < 1) the puncturing permeability rate, i.e., the fraction 
of bits surviving after puncturing, and by R' = R/X = l/(Aq) 
the nominal code rate of the punctured RMA code. 

The SIOSEF of a randomly punctured code C punct - with 
input support set size w, output support set size h before 
puncturing, and output support set size h' after puncturing 
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Asymptotic h min Growth Rate Coef 



TABLE III 

tcients for punctured RMA Code Ensembles with q = 4 and nominal code rate R' = l/(Xq). 



R' 




1/4 


0.28 


0.29 


3/10 


5/16 


0.33 


1/3 


11/30 


2/5 


0.41 


0.42 


0.43 


A 




1 


25/28 


25/29 


5/6 


4/5 


25/33 


3/4 


15/22 


5/8 


25/41 


25/42 


25/43 


Po (L = 


2) 


0.1289 


0.1192 


0.1142 


0.1077 


0.0977 


0.0819 


0.0788 


0.0474 


0.0188 


0.0112 


0.0045 


N/A 


Po (L = 


3) 


0.1194 


0.0694 


0.0528 


0.0373 


0.0198 


0.0004 


N/A 


N/A 


N/A 


N/A 


N/A 


N/A 


Pa (L = 


4) 


0.0784 


0.0112 


N/A 


N/A 


N/A 


N/A 


N/A 


N/A 


N/A 


N/A 


N/A 


N/A 



is given by [10] 

cpunct. _ Vn c 



A/ (h\ I N-h 

h'J \> 



\\N-h' 



h=h> 



LI) 



(16) 



Using Stirling's approximation in (16) and a generalization 
of Lemma 3, and coupling it with (13) and (14), the stopping 
set size spectral shape function of a punctured RMA code 
ensemble is given by 



r s (P ) = t sup — — - 

A 0</3 ,...,&<1 1 
0<7i,...,7i<l 

L 



/=i 



L 



;=i 



+ (1-/3l)H 



a(i-pT 



H(A) 



A/V 



is the normalized stopping set size after 



where p' = 
puncturing. 

The values of po corresponding to rs RMA (p') are given 
in Tables II and III for L = 2,3, and 4 mother RMA code 
ensembles with q = 3 and 4, respectively, for several nominal 
code rates R'. Asymptotic linear growth can be guaranteed for 
some rates R' > 1/q. However, it is interesting to note that the 
asymptotic stopping set size spectral shape function is strictly 
positive with heavy puncturing of the mother code ensemble, 
which implies that the asymptotic linear growth rate property 
breaks down with heavy puncturing. For instance, for L = 2 
and q = 3, the punctured ensemble remains good for the BEC 
up to rate R' = 0.54, but for heavier puncturing this property 
is lost. This phenomenon is more significant for punctured 
RMA code ensembles with a larger number of accumulators 
and for larger q. For the L = 3 and q = 3 punctured RMA 
code ensemble, the asymptotic linear growth rate property 
is lost between R' = 0.43 and R' = 4/9. In Fig. 7, we 
plot the probabilistic lower bound on h m i n from Lemma 2 
for punctured q = 4 RMA code ensembles with A — 3/4 
and L = 2 and 3. The results are in agreement with the 
asymptotic analysis; for L = 2 and permeability rate A = 3/4 
linear growth rate is guaranteed. However, when applying the 
same puncturing to the L = 3 RMA code ensemble, the 
asymptotic stopping set size spectral shape function becomes 
strictly positive, and the linear growth rate property breaks 
down. Note that these results are in contrast with the results 
in [10], where it was observed that the asymptotic normalized 
rf min gets closer to the GVB for higher rates with random 
puncturing. 



VII. Stopping Distance Analysis for HCCs 

In this section, we address the behavior of /i m j n for HCC 
ensembles in the form of Fig. 2. For brevity, we only give the 
conditional support size enumerating function (with w > 0) 
and the stopping set size spectral shape function for type-1 
and type-4 HCC ensembles. 

A. Type-1 HCC 

We consider the type-1 HCC ensemble in Fig. 2. Let q be 
the number of accumulators of the outer MPCC and let q+1 be 
the index denoting the inner accumulator. For convenience, we 
define h p = h\+- ■ -+h q , the input support set size of the inner 
accumulator C q+ \. Since none of the parallel branches of the 
outer MPCC are connected to the channel, the set Q is empty. 
Using (2) and (3), the conditional support size enumerating 
function (with w > 0) for the type-1 HCC ensemble can now 
be written as 



nq ^LfJ (K-hi\(hi-l\(hi-di 
; = lZ^d, = lV d t )\di-l)\w-2di 



,h Q ,h 



y^L 2 J ( N - h \( h ~ 1 )( h - d q +i ) 
Z~id q+ i = l \d q+1 )\d q+1 -l)\h p -2d q+1 ) 



(hj 



jL, where 



Now, let a = f , p = A = % and 7/ 
I = 1, . . . , q. We also denote the normalized input support set 
size of the inner accumulator by (3 P — and define S = 
Using Stirling's approximation and a generalization of 
Lemma 3, we get 



rf«(p)= sup I£(1- A )h(-^- 

0<«,7i,...,7,<l 



a - 2 |t; 
11 



: (a) - H (&) . 

(17) 



The numerical evaluation of (17) (the details are omitted 
due to lack of space) is given in Fig. 8 for q = 4. We remark 
that the optimization is harder than for the d m ; n case, since 
the objective function involves more variables. We observe 
that rf n (p) is strictly zero in the range (0,po = 0.1289) 
and positive for some p > po. From these results and 
the generalizations of Lemma 3 and Theorem 6 to HCC 
ensembles, p = 0.1289 is a lower bound on the asymptotic 
hmin growth rate coefficient of the ensemble. It is worth 
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mentioning that the supremum is obtained for /3i = ■ ■ ■ = (3 q 
and 71 = • • • = 7 9 , i.e., when all the accumulators of the outer 
MPCC contribute equally to the stopping set size. Therefore, 
we can set (3\ = ■ ■ ■ = /3 q = (3 P and 71 = • • • = -y q in (17), 
and the expression for the stopping set size spectral shape 
function becomes equal to (13) (with L = 2) for RAA code 
ensembles. Indeed, the type-1 HCC ensemble and the RAA 
code ensemble are identical in both asymptotic /i m ; n growth 
rate and convergence threshold (see Section VIII below). 

B. Type-4 HCC 

We consider the type-4 HCC ensemble formed by an outer 
MPCC with q constituent encoders followed by an accumu- 
lator. The first encoder (the systematic branch) performs an 
identity mapping and is sent straight through the channel 
(therefore Q = {1} in (2)). Denote the input support set size 
of the inner accumulator by h p = hi + ■ ■ ■ + h q . Also, let 
M = (q- l)K = ^j-N. Using (2) and (3), the conditional 
support size enumerating function (with w > 0) of the type-4 
HCC ensemble can be written as 



s C t 4 

b w,h 2 



,h a ,h 



nq v^LfJ (K-hi\(hi-\\(hi-di\ 
l=2/-id l = l\ d, )\d l -l)\w-2d,J 

(*r 2 

EL 2 J tM-h+w\th-w-l\(h-w-d q+1 \ 
d q +i = l\ d q+ i I V h p -2d q+ i ) 



Again, we define a 



where 



(hj 



— —In— — R, — —L 
K ' " AT' l J l K ' 

, q. Also, define the normalized size 



and 



f3 p ~ jj and 6 = Using Stirling's approximation and a 
generalization of Lemma 3, the stopping set size spectral shape 
function of the type-4 HCC ensemble can now be written as 



r C t 4 



( P )= sup 

0<a,/3 2 ,-A< 1 1 7=2 
0<<5,72,---,7g<l 
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I -Pi 



+ 



+ 



1=2 



1=2 



2li 



A - n 



a 



- P 
(1 



(q-l)S 



q - 1 - qp + a 



1)6 



P- 

q-2 



(q 



qp- 

■1)5 



(q -!)(&- 26) 



qp-a-(q - 1)6 



H(a) 



:(&)■ 



q q 

The stopping set size spectral shape functions for type- 
2 and type-3 HCCs can be obtained in a similar manner. 
The stopping set size spectral shape functions for the type-2, 
type-3, and type-4 HCC ensembles are also plotted in Fig. 8 
for q = 4 (q 1 = 1 for the type-2 HCC ensemble). The 
largest asymptotic h m i n growth rate (po = 0.1289) is obtained 
for the type-1 HCC ensemble. If one of the accumulators 
of the outer MPCC is replaced by a feedforward branch 
(type-2 HCC ensemble), the asymptotic ft, m i n growth rate is 




0.08 0.09 



0.15 0.16 



Fig. 8. Asymptotic stopping set size spectral shape function for the rate- 1/4 
HCC ensembles depicted in Fig. 2. 

TABLE IV 

Asymptotic /i min Growth Rate Coefficients for HCC Ensembles . 



PO 


q = 3 


9 = 4 


q = 5 


q = 6 


Type-1 


0.0929 


0.1289 


0.1505 


0.1647 


Type-2 


0.0716 


0.1207 


0.1462 


0.1623 


Type-3 


N/A 


0.0886 


0.1266 


0.1494 


Type-4 


N/A 


0.0829 


0.1199 


0.1430 



decreased to po = 0.1207. The asymptotic /i m ; n growth rate 
is further decreased to p — 0.0886 if the feedforward branch 
does not enter the inner accumulator. Finally, the smallest 
asymptotic /i min growth rate (p — 0.0829) is obtained for 
the fully systematic type-4 HCC ensemble. As a comparison, 
the asymptotic growth rate coefficient of the <i m ; n computed in 
[8] is 0.1911, 0.1793, 0.1350, and 0.1179 for type-1, type-2, 
type-3, and type-4 HCCs, respectively. The same behavior is 
observed for other values of q. The coefficients po for the four 
HCC ensembles with q = 3, 4, 5, and 6 are given in Table IV. 



VIII. EXIT charts Analysis 

In the previous sections, we have shown that RMA and 
HCC ensembles are good for the BEC. In this section, we 
address iterative constituent code oriented decoding of RMA 
codes and HCCs on the BEC by using EXIT charts analysis 
[29] to estimate their convergence thresholds. In particular, we 
follow the definitions of the EXIT functions in [30]. 

Denote by u = (m, . . . ,Uk) the sequence of information 
symbols which is mapped to the sequence of code symbols 
x = (xi, . . . ,xn) by an (N,K) code C. As required for 
the EXIT charts analysis, we assume that N — ► oo. The 
information symbols are transmitted over a BEC, called the 
information-symbol a priori channel, with erasure probability 
p au . For information symbol Uj, the received symbol is 
denoted by y Ui , and the corresponding a priori L- value (or 
log-likelihood ratio) by L^(m). Also, let I{ui\ L^(m)) be the 
mutual information (MI) between and L^(ui). The average 
a priori MI for the information symbols is 

K 



lZu = ^Y.l(u l ;L c a {u l ))=l 



Similarly, the code symbols are transmitted over a BEC, 
called the code-symbol a priori channel, with erasure proba- 
bility p ax . For code symbol x i7 the received symbol is denoted 
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by y Xi , and the corresponding a priori L-value by L^(xi). 
Also, let I(xi;L%(xi)) be the MI between Xi and L^(xi). 
The average a priori MI for the code symbols is 

1 N 

i=i 

The a priori L-values L^(ui) and L^(xt) are given to 
an a posteriori probability (APP) decoder, which computes 
the extrinsic L-values L^{m) and L^(xi) for information 
symbols and code symbols, respectively. The average extrinsic 
MI for information and code symbols are 

an< ^ N 

i=l 

respectively. The input-output behavior of an APP decoder 
is then completely characterized by two EXIT functions T u 
and T x which specify the evolution of the extrinsic Mis as a 
function of the a priori Mis. In particular, we can write = 
Tu{Ia,u, !a, x ) and I° x = T X (I° U , I° x ). A nice property of the 
BEC is that the EXIT functions for the repeat code and for 
convolutional encoders can be given in closed form as shown 
in [30] and [31]. 

In a concatenated coding scheme consisting of P constituent 
encoders, decoding proceeds by alternating between the cor- 
responding P constituent decoders. The iterative decoding 
process can then be tracked using a multi-dimensional EXIT 
chart [32]. Alternatively, the EXIT functions of the constituent 
decoders can be properly combined and projected into a 
two-dimensional chart [33]. For instance, convergence of the 
RMA and HCC ensembles can be visualized using a two- 
dimensional EXIT chart reporting in a single figure the EXIT 
function of the outer code (the concatenation of a repeat code 
and L — 1 accumulators for the RMA code ensemble, and 
the MPCC in the case of HCCs) and the EXIT function of 
the inner accumulator. Consider as an example the type-1 and 
type-2 HCCs of Fig. 2. As defined above, let L% >(u\) and 
L^'(x\) (resp. L^'(u\) and L^'(x\)) be the a priori (resp. 
extrinsic) L-values for the information and code symbols of 
constituent encoder Ci, respectively. Also, let I Cl , and / 1 , 
(resp. lf l u , and lf l x ,) be the corresponding Mis. The EXIT 
functions for the constituent decoders of the MPCC (i.e., 
I = 1, . . . , q) can be expressed as 

jCi rpCi I m<J jCi jCi \ 

I e,u' ~ 1 u \ < Ca i=l,ij&l 1 e,u*> I a,x') 

jCi _ rpC'i ( m<? jCi jCi \ 

where h ffl . . . ffl I n = 1 - (1 - h) • • • (1 - I n ) [34]. Note 
that I° l x , = /f^Mpgc = 1 - Pa.x^cc. The EXIT function 
of the outer MPCC I £ "mpcc can be computed for all values 
< laxSScc < 1 by activating all q decoders of the 
MPCC until I Cl , and I Cl , converge to a fixed value, and 
setting I^Sc = \i5=il%- Anally, the EXIT chart 



TABLE V 

Convergence thresholds for RMA and HCC ensembles. 





q = Z 


q = i 


q = b 


9 = 6 


Type-l/RAA 


0.4965 


0.5422 


0.5719 


0.5935 


Type-2 


0.5058 


0.5543 


0.5847 


0.6062 


Type-3 


0.5624 


0.6008 


0.6252 


0.6429 


Type -4 


0.6007 


0.6373 


0.6582 


0.6730 


RAAA 


0.3259 


0.3531 


0.3718 


0.3860 


RAAAA 


0.1957 


0.2105 


0.2209 


0.2290 



of the HCC is obtained by reporting in a single plot the 
functions J^ c (0, I c a ^ cc ) and /J^ , ), 

where I^ZSZc - I^+i and I^+U - 1 - p c h, where 
p c h is the erasure probability of the communication channel. 
The EXIT charts for RMA code ensembles and type-3 and 
type-4 HCCs can be obtained using a similar procedure. The 
convergence thresholds, i.e., the largest values of the channel 
erasure probability p ch such that there is an open tunnel in the 
EXIT charts, for RMA and HCC ensembles for q = 3, . . . , 6 
are given in Table V. From Tables V and IV the presence of a 
tradeoff between asymptotic h m i n growth rate and convergence 
threshold for HCC ensembles can be observed. In fact, the 
hierarchy arising from Table IV is completely reversed in 
Table V. The type-1 code ensemble is the best one in terms of 
asymptotic ft, m i n growth rate. However, it has the worst con- 
vergence among the four considered HCCs. The convergence 
threshold can be significantly improved if one of the parallel 
branches of the outer MPCC is sent straight through the 
channel, at the expense of a smaller asymptotic h m [ n growth 
rate coefficient. The best convergence is achieved by the fully 
systematic type-4 ensemble. On the other hand, RAAA and 
RAAAA code ensembles show very poor thresholds, which 
make them impractical. 

From the EXIT charts analysis and the asymptotic analysis 
in Sections VI and VII, it arises that double serially concate- 
nated code ensembles and HCC ensembles are good ensembles 
for the BEC, since they provide both high asymptotic /i m ; n 
growth rates and good convergence behavior, while adding 
more encoding stages in RMA codes penalizes both the 
asymptotic h m i n growth rate and the convergence threshold. 

IX. Conclusion 

In this paper, we extended the results of [10] and [11, 12], 
where RMA code ensembles were proved to be asymptotically 
good, in the sense that their typical d m i n asymptotically grows 
linearly with the block length, to show that these ensembles 
are also good for the BEC, i.e., their typical h m i n also 
grows linearly with the block length. However, contrary to the 
asymptotic d m j n , whose growth rate coefficient increases with 
the number of accumulators, the growth rate of /i m ; n decreases 
with the number of encoding stages. Therefore, double serially 
concatenated codes seem to be good for the BEC, while adding 
more encoding stages degrades performance. Furthermore, we 
considered random puncturing of the RMA code ensemble to 
achieve higher rates and showed that the asymptotic stopping 
set size spectral shape function is strictly positive for high 
rates, which implies that the asymptotic linear growth rate 
property of /i m ; n is lost when the rate is high. In particular, for 
each RMA code ensemble, i.e., for each pair L and q, there is 
a particular code rate for which the linear growth rate property 
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breaks down. This phenomenon becomes more significant for 
larger values of L and q. 

We also considered the HCC ensembles recently discussed 
in [8], and showed that they also exhibit an asymptotic h m [ n 
linear growth. A fundamental tradeoff between asymptotic 
h m in growth rate and convergence threshold was observed for 
these ensembles. In that sense, HCC ensembles offer more 
degrees of freedom for code construction than RMA code 
ensembles and therefore they allow for constructing codes 
with a better tradeoff between asymptotic /i m j n growth rate 
and iterative convergence properties. 

Appendix A 
Proof of Lemma 3 

From (11) with L = 2 we can write 

w = aN a , hi = ftN bl , h = pN c , 
di = 71 A ei , and d 2 = 72 A 62 

where < a < h < c < 1, < e 1 < a < 1, and < e 2 < 
bi < 1, and a, b\, c, e\, and e-i are not all equal to one. Also, 
a,/3i,7i,72, and p are positive constants. Now, 

JV-hi^fci-l^ /n-di ^ , Ar \ aN a 



V di Adi-lAgtu-2di/ < / N \ 



7l 7vei j y^N^ 

qaN a -2f 1 N°i 



N 



qaN a 



-qctN" 



( ftN bl -7i7V ei 
\qaN a - 271 A e i 



exp(o(A a In A)) 



(18) 



and 



pN<y^ 2 ( P N<- l2 N^ Y lNbl ~ 272NC2 



72 _/V e 2 

A 
ftN b ^ 



-/3iAf b 



exp (o(A bl In A)) 



(19) 



where we have used the inequalities 



A 



1 



< 



N 



< 



W(i-l) 



Z / </?at(Z — 1) \ Z 

from [10] to bound the binomial coefficients of the left hand 
sides of (18) and (19). The rest of the proof is similar to the 
proof of Lemma 1 in [10], where five cases are considered. 
Here, we will only consider the first two cases in [10, Eq. 
(43)] and [10, Eq. (44)] due to lack of space. The other three 
cases can be proved in a similar fashion. 

. Assume 0<a = b 1 <c<l ([10, Eq. (43)]) or < a = 
bi < c < 1 ([10, Eq. (44)]). In both cases, a = h < 1. 
Define 7^ = 71 if a = e\ and otherwise. In both cases, 
7* < since the summation index cZi in (10) is upper- 
bounded by I , from which it follows that 71 is upper- 
bounded by %y when a = e\. From (18), it follows that 



as A — ► 00, the right hand side of (18) approaches 

exp ((1 -a)(a-qa + jl)N a InN + o(N a In A)) 

< exp ((1 - a)a (l - |) A a In A + o(N a In AT)) 

— ► as A — > 00 

since we have assumed that q > 3. Define 72 = 72 if 
62 = b\ (= a) and otherwise. In both cases, 72 < 4r, 
since the summation index di in (10) is upper-bounded 
by I 4pJ , from which it follows that 72 is upper-bounded 

EL ,„ 



by ^ when e 2 = b\ ( 
it follows that as A - 
approaches 

exp((l - c)(7a - 



a). From (19), and when c < 1, 
+ 00, the right hand side of (19) 



< 



l3i)N a lnA + o(A a In A)) 
exp ^-(1 -c)yA a lnA + o(A a lnA)^ 



as A 



00. 



From (19), and when c = 1, it follows that as A — ► 00, 
the right hand side of (19) approaches exp {o{N a In A)), 
and the result of Lemma 3 is proved for the special cases 
of < a = bi < c < 1 and < a = b x < c < 1. 

Appendix B 

Maximization of the Function /(•) for RMA Code 
Ensembles 

We consider the maximization of the function 
/(/3 ,/3i, . . . ,/?l_i,7i, . . . ,7l,p) in (13). A maximum 
can occur on the boundary or in the region < 0i, 7; < 1 VZ. 
The following lemma holds. 

Lemma 4: The asymptotic stopping set size spectral shape 
function of the RMA code ensemble is non-negative, i.e., 

rf RMA (p) > 0, Vpe [0,1]. 

Proof: The values of 7; in (14) can be chosen such that 
rf RMA (/?) reduces to the spectral shape function of the code 
ensemble. Then, we can use [11, Proposition 12], and the result 
follows. ■ 
We consider first the case where the maximum occurs in 
the region < f3i, 7; < 1 VZ. In this case the maximum is 
attained at the point where all partial derivatives and J^- 
equal zero. Setting §^ = M_ = 9£_ = Q gives 



7i 



A(J3o) - B(A>)/?i 



(l-7 ; )(A-i-27 ; ) 2 -7 ; 2 ( 7; -A_i) 
7 2 + (A-i - 2 7; ) 2 



, z 



(20) 

l,...,L 
(21) 



7+i 



0?(J3, - A+i)(l 7O + A(l - ft? {ft ft-i) 



(if {I -ft- 7O + 2(1 - ft) 2 {ft + 7 i ~ ft-i) 
-- ft, 11) - D(°)(A-i,A,70A+i, 

Z = 1,...,L-1. 



(22) 
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To determine a solution to the above set of equations, we 
choose the following strategy. First treat f3 as a free pa- 
rameter and compute A(f3 Q ) and B(/3 ). Then, set 71 = 
A(Po) —B(/3o)Pi in (21) to obtain /3i by solving the resulting 
third order equation. 71 can now be obtained using (20). 
We are now in the position to compute (/3o, /3i, 71) and 
D (0) (/3 ,/3i,7i) in (22) and combine (22) and (21) to obtain 
the values for 72 and (32- Finally, the values for 7/ and /3/ with 
I > 2 can be obtained by using (22) and (21) recursively. 

We must also consider the following boundary conditions 
when they apply. 

« The boundary condition 7/ = 1 — /?/ and the condition 
If + ¥- ■ mt = result in 

dfii d'yi dpi 



_ {Pi - ft +1 ){3f + - 2(1 - f3j)f 

11+1 /3f + 2(A-i -2(1- A)) 2 

= C( 1 )(A_i,A)- J D( 1 )(/3 ; _ 1 ,A)/3m- 



(23) 



• The boundary condition 7; = /?/ and the condition + 



|£ • 2* = result in 



7i+i = 



(i-2A) 2 (A-A+i) + A(i-A) 2 



(1 - 2/3,) 2 + 2(1 - A) 2 (24) 

= c( 2 )(A-i,A)- J D (2) (A-i,A)A+i- 

• The boundary condition 7; = /3/_i — $ and the condition 
If + I 1 ' 75? = result in 

dPi d~fi dPi 
ll+l = 

Wi - A+1KA-1 - A) 2 + MM - A-i) 2 (i - A) 2 

~ A) 2 + 2(2/3/ - A_i) 2 (l - A) 2 
= c( 3 )(A_ 1 ,A)-^ (3) (A-i,A)A+i- 

• The boundary condition 7; = and the condition 

df_ , 9/ _ cha _ q resu i t ;„ 

a/3, + a 7i a/3, — u rebU1L m 
7;+i = 

ff(ft - - /3/ - %i) + /?,(! - /3/) 2 (/3/ - %i) 

/3 2 (l-/3/-%i) + 2(l-/3/)2(/3 ; -%i) 
= C (0) (/3/_i, /?/, 7/) - (/3;_i, /3/, 7; ) A+i • 

Note that for each boundary condition, Z can vary between 1 
and L for the first condition and between 1 and L — 1 for the 
second condition. For instance, when the boundary condition 
7/ = 1— fa, I = 1, . . . , L, is applied, then we substitute the two 
equations in (21) and (22) with 7/ = 1 — /3j and the equation 
in (23). Similarly, when the boundary condition 7; = /3/, 
Z = 1, . . . , L, is applied, then we substitute the two equations 
in (21) and (22) with 7/ = /3j and the equation in (24). 
In general, all possible combinations must be checked, i.e., 
applying one of the four boundary conditions or no boundary 
condition at all when computing the pair ($, 7/), I = 1, . . . , L, 
where (3^ — p. Using the procedure described above, we can 
compute the asymptotic ft min growth rate coefficient p for 
RMA code ensembles with different values of q and L, where 
po has been defined in Theorem 6. 



Appendix C 
Proof of Theorem 6 

The proof of Theorem 6 follows closely the proof of Theo- 
rem 11 in [11], which is inspired by asymptotic techniques 
devised in [35]. The proof of [11, Theorem 11] relies on 
Lemmas 26 and 27 in [11]. We start by proving [11, Lemma 
26] for the stopping set case, which is stated below in our 
notation for convenience. The lemma is proven by induction 
on L following the same arguments as in the proof for the 
codeword case outlined in [12]. 

Lemma 5: Let {/i/v}/veN be a sequence of integers such 
that for any arbitrary rj > 



Then, 



,. h N 
lim — — 



h=l 



and lim 

N >o 



In ft 



JV 



0. 



IN 



o 



(V 



where L is the number of accumulators. 

Proof: We prove the lemma by induction on the number 
of accumulators L. Consider first the case of L = 1. We have 



h=i 



2h N /q , , . h N 

£ ? E 

w=l V / h=l 



d ) (d-l) {qw-2d) 



( ) 



2h N /q 

< E NV 

W — 1 



h=i 



where (see (7)) 
9(w, N) 



(qw)\e qw+w - 1 <p N (qw - 1) L ^ J (qw - 2d) 2d - qw 
n w ll]W Z—< d d (d— l) d_1 



d=l 



Note that the upper bound of 2ft/v/g in the summation over w 
is due to the binomial ( qw l2d) • ln more detail, ft— d > qw—2d, 
from which it follows that qw<h + d<h + [qw/2\, which 
implies that w < 2h/q. Now, it follows that 



1h N jq 



JV ' I QMJ I 

N w ^g{w,N)hT m ' 2 



h=l 



W — l 

2h N 



max 

q l<w<2h N /q 

< V-rfi+vLiv)^^- 1 

for large enough N and for all rj > 0. Note that for large 

enough N, N w ~^1 dominates g(w, N)^^^^ 2 , due 
to the conditions on ft/y stated in the lemma. Now, assume 
that the statement of the lemma is true for the case of L — 1. 
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We get 

h N 



h=i 



< 



where 



-RMA(t) 



2h N 

E 

2/ijv 

E 



■"RMA(I 



"EE 



(N-h\ (h-l\ ( h-d \ 
\ d )\d-l)\w-2d) 

o 



-RMA(L-l) 



h=i 



g'(w,N) = (w)\e w - 1 (p N (w - 1 



LfJ 

E 

d=l 



(w - 2d) 



2d- 



d d (d-l) 



d-1 



and Crma(;) denotes the RMA code ensemble with I, I = 
L—l, L, accumulators. Note that the lower bound of \q/2 L ~ 1 ~\ 
in the summation over w is due to the fact that the output size 
h of an accumulator is at least [w/2l, where w is the input 
size. This is due to the binomial ( an d tne u PP er bound 

of [w/2\ in the summation over d. It follows that 



E-Crma(l) 
s h 

h=l 



< 



2h N 

E 



-RMA(L-l) 



N-\^g'(w,N)h W N 



+LfJ- 



) 



g'(w,N)h 



N 



x max iV 

= o^ 1 -^! r*i-H») 

for large enough iV and for all 77 > 0. Above, we used the 
induction hypothesis in the second inequality. Also, note that 

for large enough TV, iV - ^! dominates g'(w, N)fi^ + ^ 2 ^ 2 , 
due to the conditions on h N stated in the lemma. ■ 
We can also extend [11, Lemma 27] to the stopping set 
case. For convenience, we state this lemma below. 



Lemma 6: Let 



size spectral shape function of the RMA code ensemble, 



defined as rf RMA (p; N) = sup In 



-Crma 
'LpA/j 



Then, 



r s C — (p;N)< 2LH ^ + 1) +r^(p). 

Proof: The proof of the lemma relies on the function 
tp(u, p), defined in (15). In particular, the proof of the lemma 
is by induction on L, following the same arguments as in the 
proof for the codeword case outlined in [12], and is therefore 
omitted for brevity. ■ 
The final part of the proof of [11, Theorem 11] is also 
very general, and it can easily be extended to the stopping set 
case. In fact, the rest of the proof only relies on the following 
properties of tjj(u, p). 

• tp(u,p) is continuous; 

• p)> f° r fixed u, is strictly increasing in p < 1/2; 



• ^"' p ^ , for fixed p, is decreasing in u; and 

. lim„_ < Vp < po. 

The first three properties follow from (15), while the fourth 
property holds by assumption. Finally, by using Lemmas 5 
and 6 and the properties above, Theorem 6 is proved following 
the same arguments as in the proof of [11, Theorem 28]. 
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